<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<HTML>
<HEAD>
	<META HTTP-EQUIV="CONTENT-TYPE" CONTENT="text/html; charset=iso-8859-1">
	<TITLE></TITLE>
	<META NAME="GENERATOR" CONTENT="StarOffice 6.0  (Solaris Sparc)">
	<META NAME="AUTHOR" CONTENT=" ">
	<META NAME="CREATED" CONTENT="20020111;13083600">
	<META NAME="CHANGED" CONTENT="20020319;12001100">
	<STYLE>
	<!--
		H2 { font-family: "Sunsans Demi" }
		H3 { font-family: "Sunsans Demi" }
		BLOCKQUOTE { color: #000000 }
	-->
	</STYLE>
</HEAD>
<BODY BGCOLOR="#ffffff">
<H1><FONT COLOR="#336699"><FONT SIZE=4 STYLE="font-size: 16pt"><B>Command
Line and Scripting of Administrative Tasks in Grid Engine</B></FONT></FONT></H1>
<P STYLE="margin-bottom: 0cm">The <B>qmon(1) </B>graphical user
interface can be used to perform all administrative tasks in Grid
Engine, and its usage provides a powerful learning tool for all the
capabilities of Grid Engine. However, Grid Engine can also be
completely administered through commands issued at the shell prompt
and called from within shell scripts. Experienced administrators will
find this to be a more flexible, quicker, and powerful way to change
Grid Engine settings.</P>
<P STYLE="margin-bottom: 0cm">This HOWTO contains an overview and
examples of shell-based adminstration. In addition, it contains
additional techniques and constructions which can be used to enable
more sophisticated tasks, such as wrapper scripts. For more basic
level configuration commands, please see the HOWTO entitled &quot;<A HREF="commontasks.html">Common
Administrative Tasks</A>&quot;.</P>
<H3>Contents</H3>
<OL>
	<LI><P STYLE="margin-bottom: 0cm"><A HREF="#qconf">Add or modify
	objects using files</A></P>
	<LI><P STYLE="margin-bottom: 0cm"><A HREF="#queuehost">Modification
	of queues, hosts, and environments</A>; <A HREF="#qselect">qselect</A></P>
	<LI><P STYLE="margin-bottom: 0cm"><A HREF="#schedconf">Modification
	of global configuration and scheduler</A></P>
</OL>
<HR>
<H2><A NAME="qconf"></A>Add or modify objects using files</H2>
<P STYLE="margin-bottom: 0cm">The <B>qconf</B> command can be used to
add new objects or modify existing objects from the specification in
a file. The syntax is</P>
<BLOCKQUOTE><CODE>qconf -{A,M}&lt;object&gt; &lt;filename&gt;</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">Where <B>-A</B> means add, and <B>-M</B>
means modify.</P>
<P STYLE="margin-bottom: 0cm"><B>&lt;object&gt; </B><SPAN STYLE="font-weight: medium">can
be:</SPAN></P>
<BLOCKQUOTE><B>c: </B><SPAN STYLE="font-weight: medium">complex<BR></SPAN><B>ckpt:
</B><SPAN STYLE="font-weight: medium">checkpoint environment<BR></SPAN><B>e:
</B><SPAN STYLE="font-weight: medium">execution host<BR></SPAN><B>p:
</B><SPAN STYLE="font-weight: medium">parallel environment<BR></SPAN><B>q:
</B><SPAN STYLE="font-weight: medium">queue<BR></SPAN><B>u: </B><SPAN STYLE="font-weight: medium">user
set</SPAN></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">This option can be used in combination
with the &quot;show&quot; option of <B>qconf</B> (<B>qconf -s&lt;obj&gt;</B>)
to take an existing object, modify it, and then update the existing
object or create a new one.</P>
<H4>Example: Write a shell script to specify queues of a <SPAN STYLE="font-style: normal">checkpoint
environment</SPAN> from a list in a file</H4>
<PRE>#!/bin/sh
# ckptq.sh: specify queues of a checkpoint from a list in a file
# Usage: ckptq.sh &lt;checkpoint-env-name&gt; &lt;filename&gt;
# &lt;filename&gt; contains a list of queues,
#    separated by commas and/or newlines

TMPFILE=/tmp/ckptq.$$
CKPT=$1
QUEUELIST=$2

qconf -sckpt $CKPT | grep -v 'queue_list' &gt; $TMPFILE
echo  queue_list `cat $QUEUELIST | \
    tr &quot;\012&quot; &quot; &quot; | tr &quot;,&quot; &quot; &quot;` &gt;&gt; $TMPFILE
qconf -Mckpt $TMPFILE
rm $TMPFILE</PRE>
<HR>
<H2><A NAME="queuehost"></A>Modification of queues, hosts, and
environments</H2>
<P STYLE="margin-bottom: 0cm">Individual queues, hosts, and both
parallel and checkpointing enviroments can be modified from the
command line by using the <B>qconf -M{q,e,p,ckpt} &lt;filename&gt;</B>
command as shown above, or by using the <B>qconf -m{q,e,p,ckpt}
&lt;objname&gt;</B> command. This opens a temporary file in an
editor, and when you save any changes you make to this file and exit
the editor, the system immediately reflects those changes. However,
when you want to change many objects at once, and to change object
configuration non-interactively, the <B>qconf -<SPAN STYLE="font-style: normal">...</SPAN>attr
</B>set of commands are used. 
</P>
<P STYLE="margin-bottom: 0cm">The first type of commands makes
modifications according to the specification on the command line.</P>
<BLOCKQUOTE><CODE>qconf -{a,m,r,d}attr queue|exechost|pe|ckpt
&lt;attrib&gt; &lt;value&gt; &lt;queue_list&gt;|&lt;host_list&gt;</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">while the second makes modifications
according to specifications in a file:</P>
<BLOCKQUOTE><CODE>qconf -{A,M,R,D}attr queue|exechost<CODE>|pe|ckpt</CODE>
&lt;filename&gt;</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">In both sets of commands, the options
indicate the following:</P>
<BLOCKQUOTE><B>-A/a</B>: add attribute<BR><B>-M/m</B><SPAN STYLE="font-weight: medium">:</SPAN>
modify attribute<BR><B>-R/r</B>: replace attribute<BR><B>-D/d</B>:
delete attribute<BR><B>&lt;attrib&gt;</B>: queue or host attribute to
be changed<BR><B>&lt;value&gt;</B>: value of attribute to be
affected<BR><B>&lt;filename&gt;</B>: a file containing
attribute-value pairs</BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm"><B><I>a</I></B>, <B><I>m</I></B>, <B><I>d</I></B>
allow you to operate on individual values in a list of values, while
<B><I>r</I></B> will replace the entire list of values with the new
one which is specified, either on the command line or in the file.</P>
<H4>Examples:</H4>
<P STYLE="margin-bottom: 0cm">Change the queue type of &quot;tcf27-e019.q&quot;
to batch-only</P>
<BLOCKQUOTE><CODE>% qconf -rattr queue qtype batch tcf27-e019.q</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">Modify the queue type and shell start
behavior of tcf27-e019.q based on the contents of the file &quot;new.cfg&quot;:</P>
<BLOCKQUOTE><CODE>% cat new.cfg<BR>qtype batch interactive
checkpointing<BR>shell_start_mode unix_behavior<BR>% qconf -Rattr
queue new.cfg tcf27-e019.q</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">Attach the complexes named &quot;storage&quot;
and &quot;license&quot; to the host &quot;tcf27-e019&quot;</P>
<BLOCKQUOTE><CODE>% qconf -rattr exechost complex_list
storage,license tcf27-e019</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">Add the resource named &quot;scratch1&quot;
with a value of 1000M and &quot;long&quot; with a value of 2</P>
<BLOCKQUOTE><CODE>% qconf -rattr exechost complex_values
scratch1=1000M,long=2 tcf27-e019</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">Attach the resource named &quot;short&quot;
to the host with a value of 4</P>
<BLOCKQUOTE><CODE>% qconf -aattr exechost complex_values short=4
tcf27-e019</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">Change the value of &quot;scratch1&quot;
to 500M while leaving other values untouched</P>
<BLOCKQUOTE><CODE>% qconf -mattr exechost complex_values
scratch1=500M tcf27-e019</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">Delete the resource &quot;long&quot;</P>
<BLOCKQUOTE><CODE>% qconf -dattr exechost complex_values long
tcf27-e019</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">Add &quot;tcf27-b011.q&quot; to the
list of queues for checkpointing enviroment &quot;sph&quot;</P>
<BLOCKQUOTE><CODE>% qconf -aattr ckpt queue_list tcf27-b011.q sph</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">Change the number of slots in parallel
environment &quot;make&quot; to 50</P>
<BLOCKQUOTE><CODE>% qconf -mattr pe slots 50 make</CODE></BLOCKQUOTE>
<H3><A NAME="qselect"></A>qselect</H3>
<P STYLE="margin-bottom: 0cm">The <B>qselect</B> command outputs a
list of queues. If called with options, it lists only queues which
match the given specifications. This can be used to great advantage
in combination with the <B>qconf -...attr queue</B> commands to
target specific queues to modify.</P>
<H4>Examples:</H4>
<P STYLE="margin-bottom: 0cm">all queues on Linux machines</P>
<BLOCKQUOTE><CODE>% qselect -l arch=glinux</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">all queues on machines with 2 CPUs</P>
<BLOCKQUOTE><CODE>% qselect -l num_proc=2</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">all queues on all 4 CPU 64-bit Solaris
machines</P>
<BLOCKQUOTE><CODE>% qselect -l arch=solaris64,num_proc=4</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">queues that provide an application
license (previously configured)</P>
<BLOCKQUOTE><CODE>% qselect -l app_lic=TRUE</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">You can combine <B>qselect</B> with
<B>qconf</B> to do wide-reaching changes with a single command line.
To do this, simply put the entire <B>qselect</B> command within
backticks, and use it in place of the <B>&lt;queue_list&gt;</B> on
the <B>qconf</B> command line.</P>
<H4>Examples:</H4>
<P STYLE="margin-bottom: 0cm">set the prolog script to sol_prolog.sh
on all queues on Solaris machines</P>
<BLOCKQUOTE><CODE>% qconf -mattr queue prolog
/usr/local/scripts/sol_prolog.sh `qselect -l arch=solaris`</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">set the attribute &quot;fluent_license&quot;
to 2 on all queues on two-processor systems</P>
<BLOCKQUOTE><CODE>% qconf -mattr queue complex_values
fluent_license=2 `qselect -l num_proc=2`</CODE></BLOCKQUOTE>
<P STYLE="margin-bottom: 0cm">The use of <STRONG>qconf</STRONG> in
conjunction with <B>qselect</B> provides the most flexible way to
automate the configuration of Grid Engine queues, allowing you to
build up your own custom administration scripts.</P>
<HR>
<H2><A NAME="schedconf"></A>Modification of global configuration and
scheduler</H2>
<P STYLE="margin-bottom: 0cm">To modify the scheduler or global
configuration, the <B>qconf -m... </B>command is used, as <B>qconf
-mconf</B> to change the global configuration and <B>qconf -msconf</B>
for the scheduler. Both of these commands open up a temporary file in
an editor. When you exit the editor, any changes you have made to
this temporary file are processed by the system and take effect
immediately. The editor program used to open the temporary file is
the one specified by the <B>EDITOR</B> enviroment variable. If this
variable is undefined, then <SPAN STYLE="font-style: normal"><B>vi</B>
is used.</SPAN></P>
<P STYLE="margin-bottom: 0cm">You can take advantage of the <B>EDITOR</B>
environment variable to automate the behavior of the <B>qconf -m...</B>
commands. Change the value of this variable to point to a program
which modifies the file whose name is given by the first argument.
After this program modifies the temporary file and exits, the system
will read in the modifications and update immediately. <B>NOTE</B>:
if the modification time of the file does not change after the edit
operation, the system will sometimes incorrectly assume it has not
been modified. Therefore, there should be a &quot;sleep 1&quot;
inserted before writing the file, to ensure a different modification
time.</P>
<H4>Example: write a script to modify the schedule interval of the
scheduler</H4>
<PRE>#!/bin/ksh
# sched_int.sh: modify the schedule interval
# usage: sched_int.sh &lt;n&gt;, where &lt;n&gt; is 
# the new interval, in seconds. n &lt; 60

TMPFILE=/tmp/sched_int.$$
if [ $MOD_SGE_SCHED_INT ]; then
     grep -v schedule_interval $1 &gt; $TMPFILE
     echo &quot;schedule_interval 0:0:$MOD_SGE_SCHED_INT&quot; &gt;&gt; $TMPFILE
# sleep to ensure modification time changes
     sleep 1
     mv $TMPFILE $1
else
     export EDITOR=$0
     export MOD_SGE_SCHED_INT=$1
     qconf -msconf
fi</PRE><P STYLE="margin-bottom: 0cm">
The sample script above modifies the <B>EDITOR</B> environment to
point to itself, and then calls <B>qconf -msconf</B>. This second
nested invocation of the script then modifies the temporary file
specified by the first argument, and then exits. The Grid Engine
system then automatically reads in the changes and the first
invocation of the script terminates. The above technique can be used
in conjunction with any <B>qconf -m...</B> command. However, it is
especially useful for administration of the scheduler and global
configuration, since there is no other way to automate this.</P>
<P STYLE="margin-bottom: 0cm"><BR>
</P>
</BODY>
</HTML>

